Human Computer Interaction Methods Research Articles

Virtual Mouse

Now a days the use of hand gesture recognition became popular due to the advance technology of artificial Intelligence (AI). In recent years, there has been a growing interest in developing alternative methods for human-computer interaction (HCI) that go beyond traditional input devices like keyboards and mouse. Hand gesture recognition has emerged as a promising approach, offering intuitive and natural ways for users to interact with digital interfaces. The proposed system utilizes computer vision techniques to detect and track hand gestures in real-time. By mapping specific gestures to mouse control commands, users can navigate and interact with graphical user interfaces (GUIs) without the need for physical input devices. In this paper A hand gesture- controlled virtual mouse system utilizes the AI and Machine learning algorithms to recognize the proper hand gestures and translate them into the mouse movements. The system we are designed that the people who have problems or difficulty using a traditional mouse or keyboard it will be appropriate for them. For this we are using camera that capture images of the user’s hand, which are processed by an AI and ML algorithm to recognize the gesture of hands being made. We trained the system by using a dataset of hand gestures. Keywords — Hand gesture recognition, Machine Learning, Artificial Intelligence, Virtual Mouse, Python, Media Pipe

Spatial and temporal changes in shorebird habitats under different land use scenarios along the Yellow and Bohai Sea coasts in China

The effect of coastal wetland loss on shorebird habitat in recent years has been widely reported in previous studies. Various coastal wetland conservation and restoration measures have been implemented or will soon be implemented in China. The extent to which these measures will affect the area and structure of coastal wetland habitat in the future remains unclear. Here, we predicted changes in habitat area and structure for 39 common shorebird species along the coasts of the Yellow and Bohai Seas using a cellular automata–Markov (CA-Markov) land use scenario model and a maximum entropy species distribution model, along with terrain factors (slope, aspect, and digital evaluation model) and climate factors (temperature and precipitation) from the Data Centre for Resources and Environmental Sciences at the Chinese Academy of Sciences, land cover maps interpreted using the human–computer interactive method, and citizen science data of shorebird occurrences derived from eBird, Global Biodiversity Information Facility, and Bird Report. We found that shorebird habitat was most abundant along the coasts of Bohai Bay, Laizhou Bay, and Yancheng. The area of habitat decreased and became increasingly fragmented between 2000 and 2020 for more than half of the 39 species. Under the future business-as-usual scenario, the area of shorebird habitat decreased from 2020 to 2050, and the remaining habitat became increasingly fragmented. Under the ecological protection (EP) scenario, habitat loss was mitigated, and habitat connectivity was improved. The area of habitat was lower in 2050 under the EP scenario than in 2000 for most species, especially threatened species, suggesting that the area of habitat will not return to year-2000 levels under the EP scenario. These results emphasize the need to protect remaining shorebird habitats and implement ecological conservation measures to ensure the long-term preservation of coastal wetlands.

Inferring in-air gestures in complex indoor environment with less supervision

People have high demands for comfort and technology in indoor environments. Gestures, as a natural and friendly human computer interaction (HCI) method, have received widespread attention and have been the subject of many research studies. Traditional approaches are based on wearable devices and cameras, which can be cumbersome to operate and infringe upon users’ privacy. Millimeter-wave (mmWave) radar avoids these problems by detecting gestures in a noninvasive manner. However, it encounters practical challenges in complex indoor environments, such as dynamic disturbance from surroundings, variable usage conditions and diverse gesture patterns, which conventionally require considerable manual effort to address. In this paper, we attempt to minimize human supervision and propose a noninvasive gesture recognition method named RaGe that involves a commercial mmWave indoor radar. First, a parameter optimization framework considering gesture prior constraints is proposed for radar configuration, which functions to weaken the disturbance from surroundings. Second, we alleviate data shortages in variable usage conditions and achieve low-cost data augmentation by applying affine transformations. Third, we combine deformable convolution operations with an unsupervised attention mechanism, thus exploring the intrinsic features involved in diverse gesture patterns. Experimental results show that RaGe is able to recognize 7 gestures with 99.3% accuracy and less human supervision, surpassing the state-of-the-art methods in comparative experiments.

Classification Algorithm for fNIRS-based Brain Signals Using Convolutional Neural Network with Spatiotemporal Feature Extraction Mechanism

Brain Computer Interface (BCI) is a highly promising human–computer interaction method that can utilize brain signals to control external devices. BCI based on functional near-infrared spectroscopy (fNIRS) is considered a relatively new and promising paradigm. fNIRS is a technique of measuring functional changes in cerebral hemodynamics. It detects changes in the hemodynamic activity of the cerebral cortex by measuring oxyhemoglobin and deoxyhemoglobin (HbR) concentrations and inversely predicts the neural activity of the brain. At the present time, Deep learning (DL) methods have not been widely used in fNIRS decoding, and there are fewer studies considering both spatial and temporal dimensions for fNIRS classification. To solve these problems, we proposed an end-to-end hybrid neural network for feature extraction of fNIRS. The method utilizes a spatial–temporal convolutional layer for automatic extraction of temporally valid information and uses a spatial attention mechanism to extract spatially localized information. A temporal convolutional network (TCN) is used to further utilize the temporal information of fNIRS before the fully connected layer. We validated our approach on a publicly available dataset including 29 subjects, including left-hand and right-hand motor imagery (MI), mental arithmetic (MA), and a baseline task. The results show that the method has few training parameters and high accuracy, providing a meaningful reference for BCI development.

Interaction mode enables user perception recognition and perception optimization: An AI human-computer interaction study

Artificial intelligent (AI) has various ways of human-computer interaction, but most of them overlook the recognition of human perception. If the interaction mode is combined with psychology, the user's mood change can be identified by the user's subtle expression, movement change and voice tone change, so as to provide corresponding services and improve the user experience. Statistical analysis of human responses to different situations in cognitive psychology, incorporating them into human-computer interaction methods. The current human-computer interaction modes in products tend to be standardized, and focusing user experience on user perception will bring special experiences to users. Emotional recognition is a cross disciplinary discipline with broad application prospects, but it has not yet reached a mature stage and requires corpus enrichment, theoretical strengthening, and method innovation. The era of artificial intelligence is leading a new wave of technological progress, and emotion recognition, as an important topic in the field of artificial intelligence, can help computer intelligence recognize human emotions and make human-computer interaction more friendly. In the near future, research on emotion recognition technology will make greater progress and be better applied to practical products.

Gesture-Based Control of Multimedia Player Using Python and OpenCV

The increasing significance of computers in our daily lives, coupled with the rise of ubiquitous computing, has necessitated effective human-computer interaction. Hand gesture recognition systems have emerged as a real-time video-based solution for detecting and interpreting hand gestures, offering intelligent and natural human-computer interaction (HCI) methods. This project focuses on leveraging human hands as input devices for computer operation. Developed using Python and the OpenCV library, the program utilizes a computer webcam to capture and analyze hand shapes and patterns. The program provides real-time feedback by displaying recognized hand gestures on the live video stream. The ultimate outcome of this project is an application that enhances user experiences in contactless systems. The project recognizes and detects human hand motions using the Python computer language through a process flow that includes background subtraction, hand ROI segmentation, contour detection, and finger recognition. Techniques for processing images are used, including hand gesture detection, pattern recognition, thresholding, and contour detection. The processing of incoming photos and the creation of related keystrokes are made possible by OpenCV, a rich set of image processing tools

The development of a Modular Accessible Musical Instrument Technology Toolkit using action research

Within the field of digital musical instruments, there have been a growing number of technological developments aimed at addressing the issue of accessibility to music-making for disabled people. This study summarizes the development of one such technological system—The Modular Accessible Musical Instrument Technology Toolkit (MAMI Tech Toolkit). The four tools in the toolkit and accompanying software were developed over 5 years using an action research methodology. A range of stakeholders across four research sites were involved in the development. This study outlines the methodological process, the stakeholder involvement, and how the data were used to inform the design of the toolkit. The accessibility of the toolkit is also discussed alongside findings that have emerged from the process. This study adds to the established canon of research around accessible digital musical instruments by documenting the creation of an accessible toolkit grounded in both theory and practical application of third-wave human–computer interaction methods. This study contributes to the discourse around the use of participatory and iterative methods to explore issues with, and barriers to, active music-making with music technology. Outlined is the development of each of the novel tools in the toolkit, the functionality they offer, as well as the accessibility issues they address. The study advances knowledge around active music-making using music technology, as well as in working with diverse users to create these new types of systems.

Computer Interaction Using Hand Gestures

Abstract: Computer interaction using hand gestures is theprocess of automatically recognizing hand gestures that enablespeople to interact with the computer. Usually, interaction with a computer is done using input devices like keyboards, mice, etc., and the need to develop contactless human-computer interaction methods is the most desired feature during these pandemic days. Gesture recognition is a significant step towardsachieving contactless new types of human-computer interaction. A deep learning approach, convolution neural network, is used to recognize the hand gestures and mapped operation to the respective gesture is performed.

Automatic Assessment of Depression and Anxiety through Encoding Pupil-wave from HCI in VR Scenes

At present, there have been many studies on the methods of using the deep learning regression model to assess depression level based on behavioral signals (facial expression, speech, and language); however, the research on the assessment method of anxiety level using deep learning is absent. In this article, pupil-wave, a physiological signal collected by Human Computer Interaction (HCI) that can directly represent the emotional state, is developed to assess the level of depression and anxiety for the first time. In order to distinguish between different depression and anxiety levels, we use the HCI method to induce the participants’ emotional experience through three virtual reality (VR) emotional scenes of joyful, sad, and calm, and construct two differential pupil-waves of joyful and sad with the calm pupil-wave as the baseline. Correspondingly, a dual-channel fusion depression and anxiety level assessment model is constructed using the improved multi-scale convolution module and our proposed width-channel attention module for one-dimensional signal processing. The test results show that the MAE/RMSE of the depression and anxiety level assessment method proposed in this article is 3.05/4.11 and 2.49/1.85, respectively, which has better assessment performance than other related research methods. This study provides an automatic assessment technique based on human computer interaction and virtual reality for mental health physical examination.

To think of interdisciplinarity as intercurrence: Or, working as an interdisciplinary team to develop an ML tool to tackle online gender-based violence and hate speech

The paper reflects on the working of an interdisciplinary team consisting of researchers and activists from the field of computer science and social sciences involved in developing a user-facing, browser plug-in to detect and moderate instances of online gender-based violence, hate speech and harassment in Hindi, Indian English, and Tamil. There have been multiple calls within the field of Human-Computer Interaction (HCI) to include qualitative methods in one’s research design. These calls, while attuned to the importance of qualitative methods for HCI, ignore the intercurrent nature of different research methods, disciplines and practices. The paper borrows the concept of intercurrence from Orren & Skowronek (1996) and reorients it to explicate the practice of interdisciplinary research. It argues that intercurrence i.e. (in between, an occurrence within an occurrence) is a useful image to perceive interdisciplinarity wherein we argue that at any given point, an interdisciplinary team navigates multiple, yet simultaneously occurring temporal dimensions of differently disciplined bodies. An awareness of these multiple temporalities adds another dimension to thinking about conflicts and possibilities emerging from interdisciplinary practices and reorients interdisciplinary research towards unexpected outcomes.

Co‐designing teacher support technology for problem‐based learning in middle school science

AbstractThis paper provides an experience report on a co‐design approach with teachers to co‐create learning analytics‐based technology to support problem‐based learning in middle school science classrooms. We have mapped out a workflow for such applications and developed design narratives to investigate the implementation, modifications and temporal roles of the participants in the design process. Our results provide precedent knowledge on co‐designing with experienced and novice teachers and co‐constructing actionable insight that can help teachers engage more effectively with their students' learning and problem‐solving processes during classroom PBL implementations. Practitioner notesWhat is already known about this topic Success of educational technology depends in large part on the technology's alignment with teachers' goals for their students, teaching strategies and classroom context. Teacher and researcher co‐design of educational technology and supporting curricula has proven to be an effective way for integrating teacher insight and supporting their implementation needs. Co‐designing learning analytics and support technologies with teachers is difficult due to differences in design and development goals, workplace norms, and AI‐literacy and learning analytics background of teachers. What this paper adds We provide a co‐design workflow for middle school teachers that centres on co‐designing and developing actionable insights to support problem‐based learning (PBL) by systematic development of responsive teaching practices using AI‐generated learning analytics. We adapt established human‐computer interaction (HCI) methods to tackle the complex task of classroom PBL implementation, working with experienced and novice teachers to create a learning analytics dashboard for a PBL curriculum. We demonstrate researcher and teacher roles and needs in ensuring co‐design collaboration and the co‐construction of actionable insight to support middle school PBL. Implications for practice and/or policy Learning analytics researchers will be able to use the workflow as a tool to support their PBL co‐design processes. Learning analytics researchers will be able to apply adapted HCI methods for effective co‐design processes. Co‐design teams will be able to pre‐emptively prepare for the difficulties and needs of teachers when integrating middle school teacher feedback during the co‐design process in support of PBL technologies.

Pupil Localization Algorithm Based on Improved U-Net Network

Accurately localizing the pupil is an essential requirement of some new human–computer interaction methods. In the past, a lot of work has been done to solve the pupil localization problem based on the appearance characteristics of the eye, but these methods are often specific to the scenario. In this paper, we propose an improved U-net network to solve the pupil location problem. This network uses the attention mechanism to automatically select the contribution of coded and uncoded features in the model during the skip connection stage of the U-net network in the channel and spatial axis. It can make full use of the two features of the model in the decoding stage, which is beneficial for improving the performance of the model. By comparing the sequential channel attention module and spatial attention module, average pooling and maximum pooling operations, and different attention mechanisms, the model was finally determined and validated on two public data sets, which proves the validity of the proposed model.

A Hardware-Based Configurable Algorithm for Eye Blink Signal Detection Using a Single-Channel BCI Headset.

Eye blink artifacts in electroencephalographic (EEG) signals have been used in multiple applications as an effective method for human-computer interaction. Hence, an effective and low-cost blinking detection method would be an invaluable aid for the development of this technology. A configurable hardware algorithm, described using hardware description language, for eye blink detection based on EEG signals from a one-channel brain-computer interface (BCI) headset was developed and implemented, showing better performance in terms of effectiveness and detection time than manufacturer-provided software.

Popular post‐separation parenting smartphone apps: An evaluation

AbstractIn recent years a bewildering array of smartphone applications (“apps”) has emerged to support separated parents' communication. Post‐separation parenting apps vary in cost and features; they typically comprise a messaging tool, shared calendar, expense tracker and a means to export records for legal purposes. A key challenge for separated parents and family law practitioners alike is knowing which apps or app feature(s) can work well for different family contexts, needs and budgets. The present study sought to evaluate nine popular post‐separation parenting apps and their features using small‐n Human–Computer Interaction methods. Mediators role‐played high conflict ex‐couples while completing a set of five common post‐separation communication or organizational tasks. A cross‐case analysis of ratings was conducted. We found that (a) many of the mediators changed their apparent enthusiasm for co‐parenting apps once they had used the apps themselves; (b) all nine apps were rated somewhere between “Poor” to Fair’; and (c) features of some of the best‐known apps were not rated as highly as some of the features of more recent, lesser‐known apps.

Application of Foot Hallux Contact Force Signal for Assistive Hand Fine Control.

Accurate recognition of disabled persons' behavioral intentions is the key to reconstructing hand function. Their intentions can be understood to some extent by electromyography (EMG), electroencephalogram (EEG), and arm movements, but they are not reliable enough to be generally accepted. In this paper, characteristics of foot contact force signals are investigated, and a method of expressing grasping intentions based on hallux (big toe) touch sense is proposed. First, force signals acquisition methods and devices are investigated and designed. By analyzing characteristics of signals in different areas of the foot, the hallux is selected. The peak number and other characteristic parameters are used to characterize signals, which can significantly express grasping intentions. Second, considering complex and fine tasks of the assistive hand, a posture control method is proposed. Based on this, many human-in-the-loop experiments are conducted using human-computer interaction methods. The results showed that people with hand disabilities could accurately express their grasping intentions through their toes, and could accurately grasp objects of different sizes, shapes, and hardness using their feet. The accuracy of the action completion for single-handed and double-handed disabled individuals was 99% and 98%, respectively. This proves that the method of using toe tactile sensation for assisting disabled individuals in hand control can help them complete daily fine motor activities. The method is easily acceptable in terms of reliability, unobtrusiveness, and aesthetics.

Contactless System Navigation Using Dynamic Hand Gesture Recognition

Abstract: Hand gestures are a natural way for human-robot interaction. Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications. Traditional (or conventional) methods of human-computer interaction, namely, keyboards, pointing devices, and similar physical input tools, and the more popular and commonly used touchscreens, are only as useful through the application of pressure through physical touch. The recent pandemic has driven a major argument about the substantial and often, indispensable need for contact-less operation of systems, irrespective of the scale of the industry. Say for example, supply chain systems that are handled by scores of employees in the span of a day. It is quite fundamental knowledge now, that how public systems that work on physical contact can pose as potential vectors for diseases. Hand gesture recognition can turn out to be an excellent solution to this problem; facilitating truly contact-less interaction with computing devices, and at the same time providing dependable and respectable efficiency. This approach will be further elaborated in upcoming editions of this research article.

Motor Imagery Classification Based on EEG Sensing with Visual and Vibrotactile Guidance.

Motor imagery (MI) is a technique of imagining the performance of a motor task without actually using the muscles. When employed in a brain-computer interface (BCI) supported by electroencephalographic (EEG) sensors, it can be used as a successful method of human-computer interaction. In this paper, the performance of six different classifiers, namely linear discriminant analysis (LDA), support vector machine (SVM), random forest (RF), and three classifiers from the family of convolutional neural networks (CNN), is evaluated using EEG MI datasets. The study investigates the effectiveness of these classifiers on MI, guided by a static visual cue, dynamic visual guidance, and a combination of dynamic visual and vibrotactile (somatosensory) guidance. The effect of filtering passband during data preprocessing was also investigated. The results show that the ResNet-based CNN significantly outperforms the competing classifiers on both vibrotactile and visually guided data when detecting different directions of MI. Preprocessing the data using low-frequency signal features proves to be a better solution to achieve higher classification accuracy. It has also been shown that vibrotactile guidance has a significant impact on classification accuracy, with the associated improvement particularly evident for architecturally simpler classifiers. These findings have important implications for the development of EEG-based BCIs, as they provide valuable insight into the suitability of different classifiers for different contexts of use.

Design and Mechanical Analysis of a Cam-Linked Planetary Gear System Seedling Picking Mechanism

This study designed a cam-linked planetary gear system automatic seedling picking mechanism to address unstable operation and issues of high damage to the substrate caused by the picking mechanism of the dryland vegetable seedling transplanter. Through an analysis of the kinematic principle of the structure and the establishment of the kinematic model, computer-aided analysis software was developed using Visual Basic 6.0. Consequently, a set of structural parameter values satisfying the target trajectory was derived employing the human–computer interaction method, and the 3D model was designed. The model was imported into Adams for kinematic simulation, and the seedling picking mechanism’s trajectory during the operation was obtained through simulation. Modal analysis of the model was performed using Ansys, and the first six-order modal vibration patterns and modal frequencies of the seedling picking mechanism were obtained under the simulated working environment. The results confirmed that no resonance occurred during the operation. Comparisons of the seedling picking needle trajectory with the idle test revealed that the theoretical, simulated, and test trajectories were approximately identical. This proved the reliability of the theoretical design of the seedling picking mechanism, the machining of the parts, and the test bench construction. The success rates of seedling picking were 97.66, 96.09, 93.75, and 90.63% at 90, 100, 110, and 120 plants/min, respectively, with rates of substrate damage of 4.43%, 6.73%, 9.57%, and 14.37%, respectively. Thus, the experimental results confirmed that the operating parameters of the cam-linked planetary gear system seedling picking mechanism satisfied the design requirements.

A Review on Dynamic Hand Gesture Recognition Techniques

Abstract: Body language is one of the nonverbal methodsof communication, and it comprises hand gestures, arm movements, posturing, and gestures and facial expressions. One way to communicate information through the movement of the body is through gestures. HGR is a smart, intuitive, and easy method of human-computer interaction (HCI). HGR systems have two key applications: SLR and GBC. To help the deaf communicate with the hearing community, SLR tries to automaticallyinterpret SLs via a computer. The idea that SL is a highly ordered and primarily symbolic collection of human gestures is what led to the development of universal gesture-based HCI.

A portable SSVEP-BCI system for rehabilitation exoskeleton in augmented reality environment

In order to strengthen the participation of stroke patients in rehabilitation training and weaken the dependence of steady-state visually evoked potential (SSVEP)-based brain-computer interface (BCI) on external stimuli equipment, we build an augmented-reality-based brain-computer interface (AR-BCI) system applied to rehabilitation exoskeleton. The system uses HoloLens to design a 4-category BCI, and adopts the sequential logic decoding method to control sixteen rehabilitation movements. In offline experiments, the performance of AR-BCI is compared with computer screen-based brain-computer interface (CS-BCI), and the effects of data length, number and position of electrodes on the performance of BCI are studied. Then, the instruction classification accuracy of AR-BCI and movement accuracy of exoskeleton are evaluated in online rehabilitation training. The average recognition accuracy of AR-BCI is 90.2% in offline experiments, which is smaller gap with CS-BCI. The recognition accuracy still reaches more than 90% when only Oz and O2 electrodes are used. The online results show that the instruction classification accuracy of AR-BCI is 88.9% and the averaged information transfer rates (ITR) is 30.01 bits min−1 under the data length of 2.5 s. The movement accuracy of exoskeleton is 91.12% with the ITR of 31.63 bits min−1, which is 2.2% higher than instruction recognition accuracy of AR-BCI. These results show that AR-BCI provides a high-performance and more friendly human–computer interaction method, and greatly improves the application potential of wearable BCI.